A Task Scheduling Algorithm for Hadoop Platform

MapReduce is a kind of software framework for easily writing applications which process vast amounts of data on large clusters of commodity hardware. In order to get better allocation of tasks and load balancing, the MapReduce work mode and task scheduling algorithm of Hadoop platform is analyzed in this paper. According to this situation that the number of tasks of the smaller weight job is more, while that of the larger weight job is less, this paper introduces the idea of weighted round-robin scheduling algorithm into the task scheduling of Hadoop and puts forward the weight update rules through analyzing all the situations of weight update. Experimental result indicates that it is effective in making task allocation and achieving good balance when it is applied into the Hadoop platform which uses only JobTracker scheduling.